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hen asked about their most powerful 
learning experiences, children often 
describe opportunities to apply new 
knowledge and skills to real-world 
situations. They give examples of writing to elected 
officials to address an issue they care about in their 
communities, using math to analyze disparities 
in school discipline, and learning about coding by 
programming a robot. 


The voices of students and educators, backed up 
by decades of research on how children learn, 
are pushing educational leaders to prioritize 
instruction that fosters creativity and critical 
thinking. Across the Commonwealth, many 
educators are making instructional shifts that 
better engage students and prepare them for 
life after high school. For some, these shifts have 
been difficult. When asked what stands in their 
way, educators point to time, training, and the 
Massachusetts Comprehensive Assessment 
System (MCAS). 


The MCAS was first introduced out of a 
commitment to educational equity, and it has 
contributed to large improvements in academic 
outcomes in Massachusetts. However, the tests 
have had unintended consequences that affect 
instructional decisions in nearly all districts, 
with a disproportionate impact on economically 
disadvantaged communities and communities 
of color. 


In schools across the Commonwealth, particularly 
those concerned about facing punitive 
accountability measures because of low test 
scores, educators report that a drive towards 
success on state assessments leads them to 
devote class time to preparing students for taking 
multiple-choice tests and to focus on a narrow 
set of tested standards. Time that could be spent 
building knowledge in collaboration with peers and 
applying it to real-world situations is exchanged 
for instruction that mirrors the structure of the 
MCAS. Schools performing well on the MCAS face 
a different, but related, challenge. Concerned that 
experimenting with new instructional approaches 
will lower scores, many are hesitant to innovate 
with their instruction. 


Yet this very effect of classroom instruction 
mimicking the MCAS demonstrates that state 
assessments are a powerful lever for instructional 
change. Under the Every Student Succeeds Act 
(ESSA), states are required to administer tests 
each year in grades 3-8 and once in high school. 
ESSA provides states with significant flexibility to 
determine what these tests look like, including 
the opportunity to pilot innovative assessment 
and accountability systems such as interim and 
performance-based assessments.' This flexibility 
provides Massachusetts with an opportunity to 
design a more culturally responsive assessment 
system that prioritizes depth and application 

of learning. 


Massachusetts education stakeholders have 
engaged in a longstanding debate on the role of 
the MCAS. During a year significantly impacted 

by the pandemic, the debate around whether 

or not the state should administer the MCAS at 

all resurfaced with renewed energy. Though the 
current MCAS has significant drawbacks, it plays a 
critical role in uncovering disparities and ensuring 
resources are targeted to schools and districts 

in need of support. This brief seeks to recenter 
discussion of our K-12 accountability system, 
focusing not on whether an assessment should 
be administered, but on what an assessment 
system should look like. In particular, we explore 
the opportunity to reimagine a system that 
promotes the most effective learning strategies, 
offers actionable data for educators, and provides 
guidance on addressing inequities. 


The sections below explore the following 
questions: What does an assessment system that 
prioritizes deeper learning look like? What steps 
can Massachusetts take to move its assessment 
system in this direction? How can Massachusetts 
ensure cultural responsiveness is a core priority 
in future state assessments? Part one of the 

brief defines deeper learning and explores the 
opportunity for assessment to serve as a lever for 
instructional change. Part two looks at options 
for assessment that align with and incentivize 
deeper learning. The brief concludes with a set of 
recommended action steps for state policymakers. 


‘Interim assessments are administered throughout the year to assess a smaller set of content standards and inform 
adjustments to instruction. Performance-based assessments use authentic tasks and scenarios to assess students’ ability to 


apply the content they have learned. 


Part 1: The Opportunity 


THE CASE FOR DEEPER LEARNING 

Deeper learning allows students to engage with content in authentic ways through rigorous, 
empowering, and relevant instruction. This concept is not new. For decades, exemplary educators 
have designed learning activities that equip students with the tools to communicate clearly, work 
collaboratively, and navigate novel problems in and outside of school. Though deeper learning offers 
much promise for students, it is difficult to do well. Educators face a range of barriers to integrating 
deeper learning in their classrooms, including inadequate training, low-quality curricular materials, 
and school structures and schedules that require teachers to move quickly through a large volume 
of content. 


Following a statewide listening tour in 2018, Commissioner Jeff Riley announced a commitment 

to deeper learning as a means to address Massachusetts’ stagnant performance on national 
assessments and longstanding racial and economic opportunity gaps. The Kaleidoscope Collective 
for Learning, which provides support for schools to redesign learning experiences in alignment with 
deeper learning, represents the Commonwealth's first major initiative to incentivize this shift. 


In the largest study to date, researchers from American Institutes for Research (AIR) found students 
involved in deeper learning schools performed better on assessments of problem solving, 
reported higher levels of interpersonal and intrapersonal skills, and graduated from high schools 
and enrolled in postsecondary institutions at higher rates than those in matched schools (Bitter & 
Loney, 2015; American Institutes for Research, 2016). Coupled with students’ positive feedback on 
engaging experiences in deeper learning schools, these outcomes make the case for Massachusetts 
policymakers to strengthen their support for schools seeking to rethink instruction around 

deeper learning. 


ASSESSMENT AS A STIMULUS FOR CHANGE 

Assessment and accountability have long been seen as levers for improving education. With an early 
and fundamental goal of bringing to light, and addressing, educational inequities, Massachusetts 
has led the nation in forward-thinking assessment and accountability systems. Today, the MCAS is 
seen by many as one of the most rigorous state assessments in the country (National Center for 
Education Statistics, 2017). 


The MCAS has exposed gaps in student outcomes that have led to increased funding for under- 
resourced schools and equipped parents with the information they need to advocate for their 
children. Over time, the Commonwealth has incorporated metrics such as chronic absenteeism, 
progress towards English proficiency, and completion of advanced coursework into its school and 
district accountability system. With each addition, districts have responded by investing time and 
attention to these metrics. 


At the same time, the MCAS has significant drawbacks. Testing captures a single snapshot in time, 
and the high-stakes nature of the test has a disproportionately negative impact on students of 
color and economically disadvantaged students. Passing the 10th grade English and math tests is a 
graduation requirement in Massachusetts,? and students who do not pass are more likely than their 
peers to drop out of high school (Papay et. al., 2010). A long delay before test results are reported 


2 The OECD PISA-Based Test for Schools (PBTS) “is a digital assessment intended to help school leaders understand their 
15-year-old students’ abilities to think critically and apply their knowledge creatively in novel contexts” (OECD website). 


3 Due to the COVID-19 pandemic, the MCAS graduation requirement was temporarily suspended for the class 
of 2020, 2021, and 2022. 3 


limits the value of MCAS data to educators and school leaders. Many districts purchase private 
interim assessment systems to get the information they need to tailor instruction based on student 
performance. And, like other standardized tests, MCAS scores at the aggregate (school/district) 

level can be explained primarily by out-of-school factors including community wealth and parental 
education levels (Tienken et. al., 2016). The use of MCAS scores to “rate” districts and schools has 
stigmatized communities that perform poorly largely due to factors beyond the control of educators. 


Research on how children learn has advanced significantly in recent years. At the same time, 
technological capability has improved tremendously, allowing for new forms of assessment. As 
a result, there is an opportunity for the Commonwealth to rethink the structure of the MCAS to 
preserve the benefits of the current system while addressing its shortcomings. 


THE GROWING GAP BETWEEN STANDARDS AND 

STANDARDS-BASED ASSESSMENT 

Given the impact of standardized assessment on classroom instruction, it is imperative that 
Massachusetts measures the competencies that are most critical to students’ future success. We 
know far more about how students learn than we did when the MCAS was first administered in 1998. 
Massachusetts standards have evolved over time, with many reflecting an emphasis on application 
of knowledge and critical thinking skills. In 2011, Massachusetts incorporated the Standards for 
Mathematical Practice into its curriculum framework; these focus on building students’ abilities to 
“oroblem-solve, reason and prove, communicate, represent, make connections, conceptualize, and 
strategize in mathematics” (Massachusetts Department of Elementary and Secondary Education, 
2017). In English/Language Arts, many standards have evolved to focus on the skills that students 
will need for the future of work and civic participation. They emphasize transferable competencies 
including collaboration, media literacy, and communication. Standards have been similarly revised in 
science, with an emphasis on building deep understanding of core scientific ideas that students learn 
to apply to novel problems in the real world. These shifts towards more applied learning experiences 
and authentic instruction create challenges for standards-based assessment as it exists today. 


Though the MCAS has adapted in an attempt to measure these skills, the format of the exam 

limits the extent to which such measurement is possible. Tests consist mainly of multiple-choice 
questions, with a handful of essays, short answer, and constructed response items. This form of 
assessment cannot measure students’ ability to find and assess sources to inform research projects, 
communicate with peers to share reasoning, or collaborate with others to reach a common goal. It 
is likely that standards will continue to be revised in future years to emphasize cultural relevance 
and reflect new knowledge about how to prepare students for the evolving demands of the modern 
workplace. These areas will be particularly difficult to measure given the current test format. 


Unless changes are made to how learning is assessed, the Commonwealth risks a growing gap 
between agreed-upon priorities for student learning and what is taught in classrooms to prepare 
for state assessments. Improving alignment between assessments and evolving, twenty-first century 
standards will better incentivize classroom-level shifts toward more authentic, culturally sustaining 
instruction. At the same time, these changes will build public trust in assessment results. 
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Part 2: Looking Towards the Future 


A FRAMEWORK FOR ASSESSMENT OF DEEPER LEARNING 

A state assessment and accountability system serves multiple goals. State leaders must rethink 

the core tenets of the system in order to assess student progress in deeper learning contexts. The 
following three requirements ensure a new system provides the information needed by stakeholders 
(teachers, parents, districts, local and state government bodies, and more), while also recognizing— 
and leveraging—the influence of state assessments on instructional practice. 


Provides usable data and information for teachers, students, families, 


and policymakers. 


A state assessment and accountability system, whether tied to deeper learning or not, must 
provide information to key stakeholders. A next-generation assessment system provides the 
following key data: 


« Information for state leaders to direct resources and technical support to schools and 
districts that are struggling to meet rigorous learning targets; 


* Information for schools to support instructional improvement efforts; 


« Information for teachers to understand students’ current knowledge and identify areas 
where further support is needed; 


« Information for students to see which content they have mastered and areas in which they 
need to improve; 


« Information for families to understand how their children are performing relative to grade- 
level standards and whether their schools are meeting their expectations. 


The current year-end MCAS does not provide data to teachers quickly enough to inform instruction. 
As a result, schools and districts devote significant time and funding to fee-based assessments that 
provide the more frequent data they need. Similarly, the MCAS does not provide the information that 
students need to share ownership of their learning—a central principle of deeper learning. In order 
to co-create learning goals and understand progress, teachers and students need regular access to 
data that shows students’ current strengths and areas for growth. 


A reimagined MCAS could comprise a series of brief interim assessments rather than a single year- 
end test. Access to this form of high-quality, standards-aligned interim assessments provided by the 
state would eliminate the need to double-test students for formative and accountability purposes. In 
this model, students would complete two to three shorter assessments throughout the year, rather 
than one summative, end-of-year test. 


Other states have begun piloting the use of interim assessments for accountability purposes. 
Nebraska, for example, is currently using NWEA’s MAP Growth assessment, an adaptive, computer- 
based assessment, to monitor student learning throughout the year. The state’s goal is to provide 
usable information to teachers and create a summative score at year's end without additional testing 
(O'Keefe & Lewis, 2019; Olson & Jerald, 2020). 
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Allows students to apply their knowledge in diverse and authentic ways. 


Performance-based assessments offer an authentic means of evaluating students’ application of the 
content they have learned. Though many policymakers and practitioners are enthusiastic about the 
potential of performance-based assessment, these assessments have historically required human 
scoring, which is difficult to scale. 


States have been experimenting with performance-based assessment for accountability for many 
years. New Hampshire's Performance Assessment of Competency Education (PACE) effort, which 
launched in 2012, is the most notable example. PACE combines a mix of locally and state-designed 
performance-based assessments, administered yearly, with standardized state assessments 

that students complete once in their elementary, middle, and high school years (Latham, 2016). 
Technological advances are making performance-based assessments more scalable than they were 
when the PACE effort launched nearly a decade ago. 


Education technology providers have begun to create opportunities for students to demonstrate 
the application of skills and knowledge through computer-based performance assessments. 
Massachusetts’ pilot of anew computer-based MCAS performance assessment for science, which 
was administered in Spring 2021, offers promise. 


Promotes culturally responsive instruction and assessment. 


Historically and today, a primary aim of state standards-based assessments has been to identify, 
quantify, and address educational inequities, whether across districts and schools or within schools 
themselves. The explicit goal of the Massachusetts Education Reform Act of 1993 and later the 2001 
No Child Left Behind Act was to ensure a high-quality education for every child, based on shared 
statewide standards for students at every grade level. The MCAS and its results have contributed 

to an increase in funding to schools serving a high percentage of students of color, economically 
disadvantaged students, and English learners. At the same time, the high-stakes nature of the test, 
especially as a high school graduation requirement, has impacted economically disadvantaged 
students and students of color disproportionately. 


Rethinking assessment creates an opportunity to prioritize non-academic metrics in the state 
accountability system, such as school climate, that create additional incentives for schools to focus 
on these issues. It also allows test creators to provide opportunities for students to demonstrate 
their knowledge in ways that more closely align with their lived experiences. 


Due to technological advances, test creators now have the ability to embed student choice in 
assessment design by providing students with options for how they will demonstrate knowledge 
in a particular construct. For example, some students may elect to describe their mathematical 
thinking in writing, while others may prefer to describe their process orally. Making these changes 
requires an acknowledgement that no test is culture-free, and that all test items must provide 
details that enable students of different backgrounds to 1) see their culture represented, and 2) 
comprehend cultural content when a test item portrays a culture different from their own. 
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OPTIONS FOR ASSESSMENT AND ACCOUNTABILITY TO INCENTIVIZE 
DEEPER LEARNING 

The following table describes various types of assessment and the extent to which each option 
aligns with the principles listed previously. Each form of assessment has strengths and drawbacks 
that Massachusetts should consider in rethinking its approach to standardized testing. These 
opportunities are not mutually exclusive and could be combined to create a right-fit assessment for 


Massachusetts. 


DIFFICULTY TO PROVIDES ASSESSES PRIORITIZES 
ASSESSMENT OPTION §  inap_LEMENT USABLE DATA KNOWLEDGE CULTURAL 
APPLICATION RESPONSIVENESS 


MCAS 2.0 
(CURRENT SYSTEM) 


INTERIM 


PERFORMANCE- 
BASED 


ADDITION OF 
METRICS TO 
ACCOUNTABILITY 


HYBRID INTERIM 
& PERFORMANCE- 
BASED 


C 


CHALLENGES AND CONSTRAINTS 

The challenges of designing and implementing a new state assessment and accountability system 

are significant. While there is work underway across the country to pilot interim assessments that 

are rolled up into a summative score and to implement performance-based assessments in lieu of a 
standardized test in every grade level every year, the development of rigorous and valid approaches is 
ongoing. 


One of the challenges that must be navigated when considering interim assessments is the tension 
between state and local control over curriculum and instruction. Interim assessments, unlike a single 
cumulative assessment at year’s end, are intrinsically tied to the timing of instruction. State-approved 
interim assessments can either, in effect, mandate the scope and sequence of instruction in schools, 
or they can allow districts to choose when each assessment is administered based on local curricula, 
putting at risk the comparability of data across districts. Yet, despite these challenges, states across 
the country are experimenting with a variety of assessments and finding ways to provide usable data 
to teachers, while also rolling data up into a summative accountability system. 


Similarly, performance-based assessments allow students to demonstrate higher-order thinking, 
and they could serve as a valuable component of regularly spaced interim or year-end summative 
assessments (Darling-Hammond et al., 2013; Tung, 2010). Traditionally, the costs and time required 
to train teachers to administer performance-based assessments and evaluate proficiency with inter- 
rater reliability have been significant. Newer computer-based performance assessments offer a less 
time-intensive rating and scoring process. Massachusetts’ MCAS science pilot, which incorporates 
performance assessment, could serve as a model for other subject areas. 


Part 3: Policy Recommendations 


Massachusetts is at a critical juncture in education. The Commonwealth's assessment and 
accountability system, which once contributed to significant instructional improvement, must be 
redesigned to support the next wave of reform. We know far more about how students learn and 
develop than we did when the MCAS was first administered in the spring of 1998. Though the test 
has become more cognitively demanding over time, much of the content and structure is rapidly 
becoming outdated. The Rennie Center recommends that the Commonwealth take the following 
actions to design and pilot a computer-based performance assessment system that provides timely 
data, supports deeper learning, and prioritizes cultural responsiveness. 


PROVIDE INTERIM ASSESSMENTS FOR ACCOUNTABILITY 

By shifting from end-of-year summative assessments to a series of shorter, interim assessments, the 
MCAS can provide timely data to teachers, students, families, and school leaders, who can use it to 
make real-time adjustments to instruction. This will also support an overall decrease in testing time, 
as schools will no longer need to administer separate interim assessments to get the data they need. 


SHIFT TO A PERFORMANCE-BASED ASSESSMENT SYSTEM 

Performance-based assessments offer a powerful opportunity for students to apply their skills and 
knowledge to real-world contexts. Previously, scalability and reliability had been key concerns in 
shifting standardized testing in the direction of authentic, performance-based assessment. Though 
challenges remain, these assessment methods have become far more scalable due to advances 

in technology, with DESE’s own science assessment piloted in spring 2021 as an example. Making 

a shift towards performance assessment does not necessitate that every assessment question is 
performance-based. The Commonwealth may begin by creating a hybrid model, in which portions 
of the exam are performance-based and other portions mirror the current MCAS. 


IDENTIFY OPPORTUNITIES FOR STUDENT CHOICE IN ASSESSMENT 

When students see themselves, their context, and their culture in instruction and assessment, they 
are better able to demonstrate their knowledge (Montenegro & Jankowski, 2017). Technological 
advances are making student choice in assessment possible in ways that did not exist just a few 
years ago. For example, online platforms can give students the opportunity to show the same 
literacy skills via a choice of writing prompts or to apply the same math skill to solve different 
real-world problems that impact their community. In instances where students demonstrate the 
same skill via different questions or scenarios, an equating process will be required to adjust for 
differences in difficulty across the questions that students choose. 


DEVELOP AN OPT-IN PILOT FOR SCHOOLS 

Current instructional practices in many schools are not designed to support deeper learning. If the 
Commonwealth shifts its assessment approach, it will take time for schools, educators, and students 
to adapt to different expectations. Given the demands of this shift, a pilot assessment should be 
opt-in for schools, with significant opportunity for participating students and educators to provide 
feedback before scaling an approach statewide. Massachusetts should consider financial incentives 
for participating districts, including training opportunities for teachers and administrators to support 
the transition to deeper learning. 


SUPPORT EDUCATORS WITH NEW FORMS OF ASSESSMENT 

Incentivizing a shift to deeper learning must be paired with rich and extensive opportunities 

for teachers and school leaders to build their skills with this new pedagogical approach. The 
Commonwealth should identify opportunities for educators to learn together, in teams from schools, 
and over multiple months to roll out this work. The Department's Kaleidoscope Initiative provides a 
promising starting point for this form of statewide learning. 


Conclusion 


As a long-time leader in education, Massachusetts faces a tremendous opportunity to align its 
already rigorous state standards to a new system of assessment and accountability. Massachusetts 
has demonstrated success with pulling from the best ideas around the country and piloting new 
approaches for others to learn from. In order to be successful, the bold assessment changes 
highlighted above should be undertaken with significant input from the communities most impacted 


by the unintended consequences of the current system. By shifting its assessment practices in this 
way, the Commonwealth has the potential to make instruction more relevant, engaging and effective 
for students across the state. 
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